Opportunities for Consistent and Holistic Metrics to Support Food Systems Transformation: A Summary of a Symposium Presented at Nutrition 2023.

There is an urgent need for global food systems transformation to realize a future where planetary health reaches its full potential. Paramount to this vision is the ability of stakeholders across sectors to understand how foods and dietary patterns impact food systems inclusive of all domains of sustainability-environmental, nutrition/health, economic and social. This article is a synopsis of presentations by 3 food systems experts to share the latest science in a session entitled "How do you measure sustainability? Opportunities for consistent and holistic metrics to support food systems transformation" at the American Society for Nutrition's 2023 annual conference. As summarized here, global population data showing widespread malnutrition underscore the important role of dietary diversity through a balance of plant- and animal-source foods to achieve nutritionally adequate diets and reduce risk of noncommunicable diseases. Yet, recent international audits of countries, companies, and organizations and their sustainability targets largely demonstrate an underrepresentation of robust nutrition/health metrics to support public nutrition and health progress. Addressing limitations in diet-sustainability modeling systems provides a viable opportunity to accurately reflect the important contributions and trade-offs of diets across all domains of sustainability to ultimately support evidence-based decision making in advancing healthy food systems.

A cross-sectional survey of patient perceptions of the National Neurosurgical Audit Programme (NNAP).

BACKGROUND: The National Neurosurgical Audit Programme (NNAP) publishes mortality outcomes of consultants and neurosurgical units across the United Kingdom. It is unclear how useful outcomes data is for patients and whether it influences their decision-making process. Our aim was to identify patients' perceptions and understanding of the NNAP data and its influences. MATERIALS AND METHODS: This single-centre study was conducted in the outpatient neurosurgery clinics at a regional neurosurgical centre. All adult (age ≥ 18) neurosurgical patients, with capacity, were invited to take part. Native and non-native English speakers were eligible. Statistical analyses were performed on SPSS v28 (IBM). Ethical approval was obtained. RESULTS: A total of 84 responses were received (54.7% females). Over half (51.0%) of respondents felt that they understood a consultant's mortality outcomes. Educational level determines respondents' understanding (χ2(8) = 16.870; p = .031). Most respondents were unaware of the NNAP (89.0%). Only a third of respondents (35.1%) understood the funnel plot used to illustrate mortality. CONCLUSIONS: Most patients were unaware of the NNAP and most did not understand the data on the website. Understanding of mortality data seemed to be related to respondents' educational level which would be important to keep in mind when planning how to depict mortality data.

Antipsychotic drugs in first-episode psychosis: A target trial emulation in the FEP-CAUSAL Collaboration.

Good adherence to antipsychotic therapy helps prevent relapses in First Episode Psychosis (FEP). We used data from the FEP-CAUSAL Collaboration, an international consortium of observational cohorts to emulate a target trial comparing antipsychotics with treatment discontinuation as the primary outcome. Other outcomes included all-cause hospitalization. We benchmarked our results to estimates from EUFEST, a randomized trial conducted in the 2000s. We included 1097 patients with a psychotic disorder and less than 2 years since psychosis onset. Inverse probability weighting was used to control for confounding. The estimated 12-month risks of discontinuation for aripiprazole, first-generation agents, olanzapine, paliperidone, quetiapine, and risperidone (95% CI) were: 61.5% (52.5-70.6), 73.5% (60.5-84.9), 76.8% (67.2-85.3), 58.4% (40.4-77.4), 76.5% (62.1-88.5), and 74.4% (67.0-81.2) respectively. Compared with aripiprazole, the 12-month risk differences (95% CI) were -15.3% (-30.0, 0.0) for olanzapine, -12.8% (-25.7, -1.0) for risperidone, and 3.0% (-21.5, 30.8) for paliperidone. The 12-month risks of hospitalization were similar between agents. Our estimates support use of aripiprazole and paliperidone as first-line therapies for FEP. Benchmarking yielded similar results for discontinuation and absolute risks of hospitalization as in the original trial, suggesting that data from the FEP-CAUSAL Collaboration data sufficed to approximately remove confounding for these clinical questions.

NutriGreen image dataset: a collection of annotated nutrition, organic, and vegan food products.

Introduction: In this research, we introduce the NutriGreen dataset, which is a collection of images representing branded food products aimed for training segmentation models for detecting various labels on food packaging. Each image in the dataset comes with three distinct labels: one indicating its nutritional quality using the Nutri-Score, another denoting whether it is vegan or vegetarian origin with the V-label, and a third displaying the EU organic certification (BIO) logo. Methods: To create the dataset, we have used semi-automatic annotation pipeline that combines domain expert annotation and automatic annotation using a deep learning model. Results: The dataset comprises a total of 10,472 images. Among these, the Nutri-Score label is distributed across five sub-labels: Nutri-Score grade A with 1,250 images, grade B with 1,107 images, grade C with 867 images, grade D with 1,001 images, and grade E with 967 images. Additionally, there are 870 images featuring the V-Label, 2,328 images showcasing the BIO label, and 3,201 images without before-mentioned labels. Furthermore, we have fine-tuned the YOLOv5 segmentation model to demonstrate the practicality of using these annotated datasets, achieving an impressive accuracy of 94.0%. Discussion: These promising results indicate that this dataset has significant potential for training innovative systems capable of detecting food labels. Moreover, it can serve as a valuable benchmark dataset for emerging computer vision systems.

Benchmarking Relevance for Hospital Design and Planning: An International Web-Based Survey.

OBJECTIVE: The study aims to investigate what design practitioners and healthcare facility managers deem as important benchmarking metrics worldwide, investigating country differences in benchmarking usage and which metrics are prioritized. BACKGROUND: Benchmarking is a regular practice in the healthcare sector, both for clinical and managerial aspects to compare, measure, and improve standardized processes. However, limited knowledge is available about benchmarking procedures in hospital planning, design, and construction. METHODS: A web-based survey was designed, revised, and pilot-tested in five countries; it was adjusted according to local experts' suggestions and submitted globally via SoSci multilingual platform to persons involved in hospital design, research, construction, and facility management. It was composed of closed questions on 5-point Likert-type scale ranking frequency or importance and open-ended questions divided into six sections. Two hundred and eighty full responses have been collected. Statistical analysis was performed via PowerBI and R-Studio, while qualitative analysis was performed via MAXQDA. RESULTS: The findings reported allow for both specific insights per each country or category as well as enabling general considerations of a practice that is becoming always more international with 30%-50% of respondents working in the international context. The evaluation of the survey highlights the most important benchmarks, among others. For example, for respondents from the top five countries (Sweden, Spain, Germany, Italy, and the United States), the most important metric for benchmark comparability is whether the project was new construction, new construction attached to an existing hospital, or interior renovation. Construction date, client type (public vs. private), and country of location were also generally rated as the most important metrics by respondents. Other metrics that were consistently rated as important globally included inpatient unit layout, walking distances, number of floors, and whether all patient rooms are private. Space-related metrics are considered very important elements in the design and planning of healthcare facilities worldwide. Regarding cost-related metrics, all countries consider the ratio construction cost per building gross area as the most important. CONCLUSIONS: Benchmarking emerges as a relevant tool for hospital design and planning as it can support efficiency, standardization, and confidence; currently, benchmarking is still underutilized due to the challenge of international comparison, access to data outside each specific company, and variation design metrics nationally. Benchmarking strategies should be further investigated to support knowledge exchange and to ensure reliable and comparable information globally.

A consensus-based tool for capability benchmarking of emergency medical services in South Africa.

Introduction: Standards for Emergency Medical Services [EMS] have recently been introduced in South Africa in a movement towards the promotion of quality improvement. While these standards identify a minimum set of criteria for EMS quality they do not differentiate between services just meeting them and those exceeding them. Benchmarking may be a helpful exercise in beginning to address the question of comparative levels of capability in EMS beyond a set of minimum standards. The aim of this study was to develop a consensus-based capability benchmarking tool for EMS organizations within the South African context. Methods: A total of 12 experts in the field of EMS in South Africa consented to participate in two Delphi Surveys in order to achieve consensus on the core components of an EMS organization as well as relevant level descriptors for those components. The resulting data was used to develop a consensus-based capability benchmarking tool for EMS organizations in South Africa. Results: A consensus-based capability benchmarking tool was developed that allows organizations to distinguish whether the organization's capability, as a whole, is underdeveloped, developing, or well-developed. This is in addition to identifying how capable they are in all individual components or sub-components. Conclusion: It is recommended that further research be conducted to assess this tool's implementation within different EMS organizations in South Africa, and that this study is used as a stepping-stone for additional research into meaningful quality improvement in emergency medical services in South Africa.

A comprehensive benchmark of graph-based genetic variant genotyping algorithms on plant genomes for creating an accurate ensemble pipeline.

BACKGROUND: Although sequencing technologies have boosted the measurement of the genomic diversity of plant crops, it remains challenging to accurately genotype millions of genetic variants, especially structural variations, with only short reads. In recent years, many graph-based variation genotyping methods have been developed to address this issue and tested for human genomes. However, their performance in plant genomes remains largely elusive. Furthermore, pipelines integrating the advantages of current genotyping methods might be required, considering the different complexity of plant genomes. RESULTS: Here we comprehensively evaluate eight such genotypers in different scenarios in terms of variant type and size, sequencing parameters, genomic context, and complexity, as well as graph size, using both simulated and real data sets from representative plant genomes. Our evaluation reveals that there are still great challenges to applying existing methods to plants, such as excessive repeats and variants or high resource consumption. Therefore, we propose a pipeline called Ensemble Variant Genotyper (EVG) that can achieve better genotyping performance in almost all experimental scenarios and comparably higher genotyping recall and precision even using 5× reads. Furthermore, we demonstrate that EVG is more robust with an increasing number of graphed genomes, especially for insertions and deletions. CONCLUSIONS: Our study will provide new insights into the development and application of graph-based genotyping algorithms. We conclude that EVG provides an accurate, unbiased, and cost-effective way for genotyping both small and large variations and will be potentially used in population-scale genotyping for large, repetitive, and heterozygous plant genomes.

Effect of mortality prediction models on resource use benchmarking of intensive care units.

PURPOSE: Intensive care requires extensive resources. The ICUs' resource use can be compared using standardized resource use ratios (SRURs). We assessed the effect of mortality prediction models on the SRURs. MATERIALS AND METHODS: We compared SRURs using different mortality prediction models: the recent Finnish Intensive Care Consortium (FICC) model and the SAPS-II model (n = 68,914 admissions). We allocated the resources to severity of illness strata using deciles of predicted mortality. In each risk and year stratum, we calculated the expected resource use per survivor from our modelling approaches using length of ICU stay and Therapeutic Intervention Scoring System (TISS) points. RESULTS: Resource use per survivor increased from one length of stay (LOS) day and around 50 TISS points in the first decile to 10 LOS-days and 450 TISS in the tenth decile for both risk scoring systems. The FICC model predicted mortality risk accurately whereas the SAPS-II grossly overestimated the risk of death. Despite this, SRURs were practically identical and consistent. CONCLUSIONS: SRURs provide a robust tool for benchmarking resource use within and between ICUs. SRURs can be used for benchmarking even if recently calibrated risk scores for the specific population are not available.

Knowledge sharing in manufacturing using LLM-powered tools: user study and model benchmarking.

Recent advances in natural language processing enable more intelligent ways to support knowledge sharing in factories. In manufacturing, operating production lines has become increasingly knowledge-intensive, putting strain on a factory's capacity to train and support new operators. This paper introduces a Large Language Model (LLM)-based system designed to retrieve information from the extensive knowledge contained in factory documentation and knowledge shared by expert operators. The system aims to efficiently answer queries from operators and facilitate the sharing of new knowledge. We conducted a user study at a factory to assess its potential impact and adoption, eliciting several perceived benefits, namely, enabling quicker information retrieval and more efficient resolution of issues. However, the study also highlighted a preference for learning from a human expert when such an option is available. Furthermore, we benchmarked several commercial and open-sourced LLMs for this system. The current state-of-the-art model, GPT-4, consistently outperformed its counterparts, with open-source models trailing closely, presenting an attractive option given their data privacy and customization benefits. In summary, this work offers preliminary insights and a system design for factories considering using LLM tools for knowledge management.

The Getting It right First Time (GIRFT) programme in urology; rationale and methodology.

The Getting It Right First Time (GIRFT) programme is a quality improvement initiative covering the National Health Service in England. The programme aims to standardise clinical practices and improve patient and system level outcomes by utilising data-driven insights and clinically-led recommendations. There are GIRFT workstreams for every medical and surgical specialty, including urology. Defining features of the GIRFT methodology are that it is clinically led by experienced clinicians, data-driven, and specialty specific. Each specialty workstream conducts deep-dive visits to every hospital, analysing performance data and engaging with clinicians and management to identify and share improvement priorities. For urology, GIRFT has completed deep-dive visits and published reports outlining priority areas for development. Reports include recommendations pertaining to streamlining care pathways, reducing the acuity of care environments, enhancing emergency services, optimising utilisation of outpatient services, and workforce training and utilisation. The GIRFT academy provides guides for implementing best practices specific to priority areas of care. These include important disease pathways, and GIRFT-advocated innovations such as urology investigation units and urology area networks. GIRFT offers clinical transformation, cost reduction, equity in access to care, and leaner models of care that are often more environmentally sustainable. Evaluation efforts of the programme have focussed on assessing the adoption of GIRFT recommendations, understanding barriers to change, and modelling the climate impact of advocated practices.

Conducting a National RT-QI Project - Challenges and Opportunities.

Over the past decade, there has been an increased interest in defining and monitoring quality indicators (QI) in the field of oncology including the field of radiation oncology. The comprehensive gathering and analysis of QIs on a multicentric scale offer valuable insights into identifying gaps in clinical practice and fostering continuous improvement. This article delineates the evolution and results of the Belgian national project dedicated to radiotherapy-specific QIs while also exploring the challenges and opportunities inherent in implementing such a multi-centric initiative.

An overview of log likelihood ratio cost in forensic science - Where is it used and what values can we expect?

There is increasing support for reporting evidential strength as a likelihood ratio (LR) and increasing interest in (semi-)automated LR systems. The log-likelihood ratio cost (Cllr) is a popular metric for such systems, penalizing misleading LRs further from 1 more. Cllr = 0 indicates perfection while Cllr = 1 indicates an uninformative system. However, beyond this, what constitutes a "good" Cllr is unclear. Aiming to provide handles on when a Cllr is "good", we studied 136 publications on (semi-)automated LR systems. Results show Cllr use heavily depends on the field, e.g., being absent in DNA analysis. Despite more publications on automated LR systems over time, the proportion reporting Cllr remains stable. Noticeably, Cllr values lack clear patterns and depend on the area, analysis and dataset. As LR systems become more prevalent, comparing them becomes crucial. This is hampered by different studies using different datasets. We advocate using public benchmark datasets to advance the field.

Results of the 2023 ISBI challenge to reduce GABA-edited MRS acquisition time.

PURPOSE: Use a conference challenge format to compare machine learning-based gamma-aminobutyric acid (GABA)-edited magnetic resonance spectroscopy (MRS) reconstruction models using one-quarter of the transients typically acquired during a complete scan. METHODS: There were three tracks: Track 1: simulated data, Track 2: identical acquisition parameters with in vivo data, and Track 3: different acquisition parameters with in vivo data. The mean squared error, signal-to-noise ratio, linewidth, and a proposed shape score metric were used to quantify model performance. Challenge organizers provided open access to a baseline model, simulated noise-free data, guides for adding synthetic noise, and in vivo data. RESULTS: Three submissions were compared. A covariance matrix convolutional neural network model was most successful for Track 1. A vision transformer model operating on a spectrogram data representation was most successful for Tracks 2 and 3. Deep learning (DL) reconstructions with 80 transients achieved equivalent or better SNR, linewidth and fit error compared to conventional 320 transient reconstructions. However, some DL models optimized linewidth and SNR without actually improving overall spectral quality, indicating a need for more robust metrics. CONCLUSION: DL-based reconstruction pipelines have the promise to reduce the number of transients required for GABA-edited MRS.

Benchmarking System Monitoring on Quality Improvement in Percutaneous Coronary Intervention: A Nationwide Registry in Japan.

Background: Quality indicators (QIs) have been developed to improve and standardize care quality in percutaneous coronary intervention (PCI). In Japan, consecutive PCI procedures are registered in a nationwide database (the Japanese Percutaneous Coronary Intervention registry), which introduces a benchmarking system for comparing individual institutional performance against the national average. Objectives: The aim of this study was to assess the impact of the benchmarking system implementation on QI improvement at the hospital level. Methods: A total of 734,264 PCIs were conducted at 1,194 institutions between January 2019 and December 2021. In January 2018, a web-based benchmarking system encompassing 7 QIs for PCI at the institutional level, including door-to-balloon time and rate of transradial intervention, was introduced. The process by which institutions tracked their QIs was centrally monitored. Results: During the 3-year study period, the benchmarking system was reviewed at least once at 742 institutions (62.1%) (median 4 times; Q1-Q3: 2-7 times). The institutions that reviewed their records had higher PCI volumes. Among these institutions, although door-to-balloon time was not directly associated, the proportion of transradial intervention increased by 2.3% in the system review group during the initial year compared with 0.7% in their counterparts. However, in the subsequent year, the association between system reviews and QI improvement was attenuated. Conclusions: The implementation of a benchmarking system, reviewed by participating institutions in Japan, was partially associated with improved QIs during the first year; however, this improvement was attenuated in the subsequent year, highlighting the need for further efforts to develop effective and sustainable interventions to enhance care quality in PCI.

Development and internal validation of an algorithm for estimating mortality in patients encountered by physician-staffed helicopter emergency medical services.

BACKGROUND: Severity of illness scoring systems are used in intensive care units to enable the calculation of adjusted outcomes for audit and benchmarking purposes. Similar tools are lacking for pre-hospital emergency medicine. Therefore, using a national helicopter emergency medical services database, we developed and internally validated a mortality prediction algorithm. METHODS: We conducted a multicentre retrospective observational register-based cohort study based on the patients treated by five physician-staffed Finnish helicopter emergency medical service units between 2012 and 2019. Only patients aged 16 and over treated by physician-staffed units were included. We analysed the relationship between 30-day mortality and physiological, patient-related and circumstantial variables. The data were imputed using multiple imputations employing chained equations. We used multivariate logistic regression to estimate the variable effects and performed derivation of multiple multivariable models with different combinations of variables. The models were combined into an algorithm to allow a risk estimation tool that accounts for missing variables. Internal validation was assessed by calculating the optimism of each performance estimate using the von Hippel method with four imputed sets. RESULTS: After exclusions, 30 186 patients were included in the analysis. 8611 (29%) patients died within the first 30 days after the incident. Eleven predictor variables (systolic blood pressure, heart rate, oxygen saturation, Glasgow Coma Scale, sex, age, emergency medical services vehicle type [helicopter vs ground unit], whether the mission was located in a medical facility or nursing home, cardiac rhythm [asystole, pulseless electrical activity, ventricular fibrillation, ventricular tachycardia vs others], time from emergency call to physician arrival and patient category) were included. Adjusted for optimism after internal validation, the algorithm had an area under the receiver operating characteristic curve of 0.921 (95% CI 0.918 to 0.924), Brier score of 0.097, calibration intercept of 0.000 (95% CI -0.040 to 0.040) and slope of 1.000 (95% CI 0.977 to 1.023). CONCLUSIONS: Based on 11 demographic, mission-specific, and physiologic variables, we developed and internally validated a novel severity of illness algorithm for use with patients encountered by physician-staffed helicopter emergency medical services, which may help in future quality improvement.

Single-cell Profiling of Reprogrammed Human Neural Stem Cells Unveils High Similarity to Neural Progenitors in the Developing Central Nervous System.

BACKGROUND: Similar to induced pluripotent cells (iPSCs), induced neural stem cells (iNSCs) can be directly converted from human somatic cells such as dermal fibroblasts and peripheral blood monocytes. While previous studies have demonstrated the resemblance of iNSCs to neural stem cells derived from primary sources and embryonic stem cells, respectively, a comprehensive analysis of the correlation between iNSCs and their physiological counterparts remained to be investigated. METHODS: Nowadays, single-cell sequencing technologies provide unique opportunities for in-depth cellular benchmarking of complex cell populations. Our study involves the comprehensive profiling of converted human iNSCs at a single-cell transcriptomic level, alongside conventional methods, like flow cytometry and immunofluorescence stainings. RESULTS: Our results show that the iNSC conversion yields a homogeneous cell population expressing bona fide neural stem cell markers. Extracting transcriptomic signatures from published single cell transcriptomic atlas data and comparison to the iNSC transcriptome reveals resemblance to embryonic neuroepithelial cells of early neurodevelopmental stages observed in vivo at 5 weeks of development. CONCLUSION: Our data underscore the physiological relevance of directly converted iNSCs, making them a valuable in vitro system for modeling human central nervous system development and establishing translational applications in cell therapy and compound screening.

Benchmarking multi-omics integration algorithms across single-cell RNA and ATAC data.

Recent advancements in single-cell sequencing technologies have generated extensive omics data in various modalities and revolutionized cell research, especially in the single-cell RNA and ATAC data. The joint analysis across scRNA-seq data and scATAC-seq data has paved the way to comprehending the cellular heterogeneity and complex cellular regulatory networks. Multi-omics integration is gaining attention as an important step in joint analysis, and the number of computational tools in this field is growing rapidly. In this paper, we benchmarked 12 multi-omics integration methods on three integration tasks via qualitative visualization and quantitative metrics, considering six main aspects that matter in multi-omics data analysis. Overall, we found that different methods have their own advantages on different aspects, while some methods outperformed other methods in most aspects. We therefore provided guidelines for selecting appropriate methods for specific scenarios and tasks to help obtain meaningful insights from multi-omics data integration.

Extend the benchmarking indel set by manual review using the individual cell line sequencing data from the Sequencing Quality Control 2 (SEQC2) project.

Accurate indel calling plays an important role in precision medicine. A benchmarking indel set is essential for thoroughly evaluating the indel calling performance of bioinformatics pipelines. A reference sample with a set of known-positive variants was developed in the FDA-led Sequencing Quality Control Phase 2 (SEQC2) project, but the known indels in the known-positive set were limited. This project sought to provide an enriched set of known indels that would be more translationally relevant by focusing on additional cancer related regions. A thorough manual review process completed by 42 reviewers, two advisors, and a judging panel of three researchers significantly enriched the known indel set by an additional 516 indels. The extended benchmarking indel set has a large range of variant allele frequencies (VAFs), with 87% of them having a VAF below 20% in reference Sample A. The reference Sample A and the indel set can be used for comprehensive benchmarking of indel calling across a wider range of VAF values in the lower range. Indel length was also variable, but the majority were under 10 base pairs (bps). Most of the indels were within coding regions, with the remainder in the gene regulatory regions. Although high confidence can be derived from the robust study design and meticulous human review, this extensive indel set has not undergone orthogonal validation. The extended benchmarking indel set, along with the indels in the previously published known-positive set, was the truth set used to benchmark indel calling pipelines in a community challenge hosted on the precisionFDA platform. This benchmarking indel set and reference samples can be utilized for a comprehensive evaluation of indel calling pipelines. Additionally, the insights and solutions obtained during the manual review process can aid in improving the performance of these pipelines.

Comment on "A novel super-enhancer-related gene signature predicts prognosis and immune microenvironment for breast cancer".

The primary aim of this study is to critically evaluate and comment on the research presented in the article titled "A Novel Super-Enhancer-Related Gene Signature Predicts Prognosis and Immune Microenvironment for Breast Cancer" by Wu et al. Our specific objectives include assessing the methodology employed by the authors, particularly in regard to the utilization of a super-enhancer-related gene signature for breast cancer prognosis prediction. We propose the necessity of subgroup analysis to effectively address the heterogeneity in breast cancer subtypes, which is crucial for the applicability of the SERGs across diverse breast cancer cases. Additionally, we suggest conducting a more comprehensive immune panel study to deepen the understanding of how the immune microenvironment impacts breast cancer prognosis. Our commentary seeks to provide valuable insights into the strengths and weaknesses of the study, contributing to a more comprehensive understanding of its findings and potential clinical implications.